07. IoU Quiz

For the next quiz we will calculate the IOU metric given ground truth and prediction matrices. Remember IOU is Intersection over Union, where the Intersection set is an AND operation (pixels that are truly part of a class AND are classified as part of the class by the network) and the Union is an OR operation (pixels that are truly part of that class + pixels that are classified as part of that class by the network).

IoU Value Quiz

QUESTION:

What is the mean IoU of the following ground truth and prediction? Since the Intersection will always be <= Union, the final answer is a value between 0 and 1. 

# ground truth
[[0, 0, 0, 0] 
 [1, 1, 1, 1] 
 [2, 2, 2, 2] 
 [3, 3, 3, 3]]

# prediction
[[0, 0, 0, 0] 
 [1, 0, 0, 1]
 [1, 2, 2, 1] 
 [3, 3, 0, 3]]

SOLUTION:

NOTE: The solutions are expressed in RegEx pattern. Udacity uses these patterns to check the given answer

TensorFlow IoU

Let's look at the tf.metrics.mean_iou function. Like all the other TensorFlow metric functions , it returns a Tensor for the metric result and a Tensor Operation to generate the result. In this case it returns mean_iou for the result and update_op for the update operation. Make sure to run update_op before getting the result from mean_iou . 

sess.run(update_op)
sess.run(mean_iou)

The other characteristic of TensorFlow metric functions is the usage of local TensorFlow variables . These are temporary TensorFlow Variables that must be initialized by running tf.local_variables_initializer() . This is similar to tf.global_variables_initializer() , but for different TensorFlow Variables.

IoU Quiz

In this quiz, you'll use this documentation to apply mean IoU to an example prediction.

Start Quiz: 

import oldtensorflow as tf


def mean_iou(ground_truth, prediction, num_classes):
    # TODO: Use `tf.metrics.mean_iou` to compute the mean IoU.
    iou, iou_op = None
    return iou, iou_op


ground_truth = tf.constant([
    [0, 0, 0, 0], 
    [1, 1, 1, 1], 
    [2, 2, 2, 2], 
    [3, 3, 3, 3]], dtype=tf.float32)
prediction = tf.constant([
    [0, 0, 0, 0], 
    [1, 0, 0, 1], 
    [1, 2, 2, 1], 
    [3, 3, 0, 3]], dtype=tf.float32)
    
# TODO: use `mean_iou` to compute the mean IoU
iou, iou_op = mean_iou()

with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        # need to initialize local variables for this to run `tf.metrics.mean_iou`
        sess.run(tf.local_variables_initializer())
        
        sess.run(iou_op)
        # should be 0.53869
        print("Mean IoU =", sess.run(iou))
import oldtensorflow as tf


def mean_iou(ground_truth, prediction, num_classes):
    # TODO: Use `tf.metrics.mean_iou` to compute the mean IoU.
    iou, iou_op = tf.metrics.mean_iou(ground_truth, prediction, num_classes)
    return iou, iou_op


ground_truth = tf.constant([
    [0, 0, 0, 0], 
    [1, 1, 1, 1], 
    [2, 2, 2, 2], 
    [3, 3, 3, 3]], dtype=tf.float32)
prediction = tf.constant([
    [0, 0, 0, 0], 
    [1, 0, 0, 1], 
    [1, 2, 2, 1], 
    [3, 3, 0, 3]], dtype=tf.float32)
    
# TODO: use `mean_iou` to compute the mean IoU
iou, iou_op = mean_iou(ground_truth, prediction, 4)

with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        # need to initialize local variables for this to run `tf.metrics.mean_iou`
        sess.run(tf.local_variables_initializer())
        
        sess.run(iou_op)
        # should be 0.53869
        print("Mean IoU =", sess.run(iou))

Next Concept